Synthesizing Adaptive Test Strategies from Temporal Logic Specifications

Constructing good test cases is difficult and time-consuming, especially if the system under test is still under development and its exact behavior is not yet fixed. We propose a new approach to compute test strategies for reactive systems from a given temporal logic specification using formal methods. The computed strategies are guaranteed to reveal certain simple faults in every realization of the specification and for every behavior of the uncontrollable part of the system's environment. The proposed approach supports different assumptions on occurrences of faults (ranging from a single transient fault to a persistent fault) and by default aims at unveiling the weakest one. Based on well-established hypotheses from fault-based testing, we argue that such tests are also sensitive for more complex bugs. Since the specification may not define the system behavior completely, we use reactive synthesis algorithms with partial information. The computed strategies are adaptive test strategies that react to behavior at runtime. We work out the underlying theory of adaptive test strategy synthesis and present experiments for a safety-critical component of a real-world satellite system. We demonstrate that our approach can be applied to industrial specifications and that the synthesized test strategies are capable of detecting bugs that are hard to detect with random testing

Fifty Shades of Grey in SOA Testing

Abstract-Testing is undisputedly a fundamental verification principle in the software landscape. Today's products require us to effectively handle and test huge, complex systems and in this context to tackle challenging traits like heterogeneity, distribution and controllability to name just a few. The advent of ServiceOriented Architectures with their inherent technological features like dynamics and heterogeneity exacerbated faced challenges, requiring us to evolve our technology. The traditional view of white or black box testing, for example, does not accommodate the multitude of shades of grey one should be able to exploit effectively for system-wide tests. Today, while there are a multitude of approaches for testing single services, there is still few work on methodological system tests for SOAs. In this paper we propose a corresponding workflow for tackling SOA testing and diagnosis, discuss SOA test case generation in more detail, and report preliminary research in that direction

Minimal hitting set computation via hypothesis exploration

Minimal hitting set (MHS) computation is a challenging problem in conflict-oriented model-based diagnosis. This paper is a first attempt to face the problem by searching the powerset H of the conflicts’ domain, which exhibits a partial order in respect of subset inclusion. Our idea is to enhance this partial order by ordering each h in H, called a hypothesis, in each (cardinality) layer of the powerset. The overall regularity can then be exploited to process a layer at a time, according to an ascending cardinality of the hypotheses, checking whether each hypothesis is valid, and pruning all its supersets (that necessarily belong to the next layers) in one shot if it is. This is a high-level description of a template anytime algorithm that can host different methods of checking whether a given hypothesis is valid, and that is the basis not only for a monolithic computation of MHSs but also for a distributed one

BPEL Integration Testing

Abstract. Service-oriented architectures, and evolvements such as clouds, provide a promising infrastructure for future computing. They encapsulate an IP core's functionality for easy access via well-defined business and web interfaces, and in turn allow us to flexibly realize complex software drawing on available expertise. In this paper, we take a look at some challenges we have to face during the task of testing such systems for verification purposes. In particular, we delve into the task of test suite generation, and compare the performance of two corresponding algorithms. In addition, we report on experiments for a collection of BPEL processes taken from the literature, in order to identify performance trends with respect to fault coverage metrics. Our results suggest that a structural reasoning might outperform a completely random approach

Symbolic Implementation of Alternating Automata

We show how to convert alternating B¨uchi automata to symbolic structures, using a variant of Miyano and Hayashi’s construction. We avoid building the nondeterministic equivalent of the alternating automaton, thus save an exponential factor in space. For one-weak automata, Miyano and Hayashi’s approach produces automata that are larger than needed. We show a hybrid approach that produces a smaller nondeterministic automaton if part of the alternating automaton is one weak. We perform a thorough experimental analysis and conclude that the symbolic approach outperforms the explicit one